Efficient Algorithms for Learning to Play Repeated Games Against Computationally Bounded Adversaries

In the game theory literature, there is an intriguing line of research on the problem of playing a repeated matrix game against an adversary whose computational resources are limited in some way. Perhaps the main way in which this research differs from classical game theory lies in the fact that when our adversary is not playing the minimax optimal strategy for the game, we may be able to attain payoff that is significantly greater than the minimax optimum. In this situation, the correct measure of our performance is in comparison to the optimum achievable against the particular adversary, not to the minimax optimum. The typical approach is to assume that the adversary’s strategy is a member of some natural class of computationally bounded strategies — most often, a class of finite automata. (For a survey on the area of “bounded rationality”, see the paper of Kalai [4].) Many previous papers examine how various aspects of classical game theory change in this setting; a good example is the question of whether cooperation is a stable solution for prisoner’s dilemma when both players are finite automata [6, 8]. Some authors have examined the further problem of learning to play optimally against an adversary whose precise strategy is unknown, but is constrained to lie in some known class of strategies (for instance, see Gilboa and Samet [3]). It is this research that forms our starting point. The previous work on learning to play optimally usually does not explicitly take into account the computational efficiency of the learning algorithm, and often gives algorithms whose running time is exponential in some natural measure of the adversary’s complexity; a notable recent exception is the work